Machine Ethics, the Frame Problem, and Theory of Mind

Work in machine ethics has thus far been focused on giving autonomous agents the ability to select morally-correct behaviors given well-formed moral problems. While this is a necessary component to enable an agent to comport to standards of moral behavior, it is not sufficient. In this paper, we present a simple task-domain to illustrate this point. We show that even in simple domains, the potential for deception and trickery on the part of the humans interacting with morally-sensitive agents will require these agents to have sophisticated cognitive faculties in order to avoid unethical behavior.